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An overall indicator, or figure of merit (FOM), for the quality of pilot performance is 
needed to define "optimal" workload levels, predict system failure, measure the impact of 
new automation in the cockpit, and define the relative contributions of subtasks to overall 
task . performance. A normative FOM has been developed (ref. 1) based on the calculation of 
a standard score for each component of a complex task. It reflected some effects, detailed in 
an earlier study (ref. 2), of the introduction of new data link technology into the cockpit. 

Since the technique showed promise, further testing was done this summer. 

A new set of data was obtained using the recently developed Multi-Attribute Task Battery 
(ref. 3). This is a complex battery consisting of four tasks which can be varied in task 

demand, and on which performance measures can be obtained. It is illustrated in Table 1. 


Tabic 1. Tasks in the Multi- Attribute Task Battery, with methods of controlling task demand 
and performance measures. 


Task 


Demand 

PtfrfflrmaaoL- 

measure 

Monitoring 

changes in lights and 
dials 

events per minute 

response lime 
onset 

to event 

Tracking 

2-dimensional, first 
order compensatory 
task 

frequency of 
generating function 

RMS error 


Com municalions 

responses to verbal 
messages 

events per minute 

response time 
unset 

to event 

Resou rcc 
Management 

adjusting fuel level in 
6 tanks with 8 pumps 

ratio of pump flow 
rales 

RMS error from 2500 
gals 


This battery was presented to 12 subjects in a 20 minute trial at each of three levels of 
workload or task demand, and performance measures collected on all four tasks. The NASA- 
TLX workload rating scale was presented at minutes 6, 12, and 18 of each trial. A figure of 
merit was then obtained for each run of the battery by calculating a mean, SD, and standard 
score (number of SD units away from the mean) for each task. This procedure, with its 
rationale, is described in more detail in reference 1. 

The resulting figure of merit increased significantly with increasing workload and was 
also positively correlated with error rate in the monitoring task, so that, when the FOM 
indicated poorer performance, missed signals were also more likely. 

Each task contributed its own proportion to the overall F'OM, and relative contributions 
changed with increasing workload. Figure 1 shows decreases in performance on tracking 
and resource management, but not on communications and monitoring, when workload 
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increases. Figure 2 shows ihe increase in resources (hat had to be dcvoied to 
communications and monitoring in order to maintain that constant performance, and that 
this was at the expense of performance on tracking and resource management. Thus, the 
FOM shows the effect of task changes, not only on the individual task that is changed (e.g. 
obviated by automation or greatly increased by a near accident), but also on the 
performance of other tasks and of the whole task. The cost to other tasks of maintaining 
constant performance on an individual task can be quantified. 

The ratings collected later in the task got lower under low workload and higher under 
high workload, i.c., easy tasks got easier with lime, while hard tasks got harder. 
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Figure 1 . Whole task and sublusk performance at 
three workload levels 
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Figure 2. Proportion of cognitive resources used by four time shared 
tasks under three workload levels 
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Abstract 


While Director of the 1990 LARSS program, I designed a Survey for Langley 
Aerospace Summer Scholars. The main purposes of the survey were to track 
those students who participated in LARSS. The objectives included tracking 
those continuing their education, and those permanently employed in industry, 
government, and higher education, and creating a database for future tracking. 

One of the most significant results is that there are currently 26 past LARSS 
graduates currently employed by NASA or NASA Contractors. 

Of the responses, 62% indicate that they are continuing their education with 65% 
enrolled in graduate programs and 35% enrolled in undergraduate programs. Of 
these, 22% are pursuing doctoral degrees, 43% are pursuing masters, and 35% 
are bachelor level students. 

It is also significant that 49% of those permanently employed are working for the 
government or a federal research laboratory; 47% are working in industry, and 
5% are working in higher education. Eighty-one per cent of those working for the 
government are NASA employees or NASA Contractor employees. 

The following is a synopsis of the data obtained from the responses: 


Took £_ 

Responses 

Percentage 

Surveys Sent 

197 



Surveys Returned 

134 


68% 

Graduates Continuing Education 

83 


62% 

Bachelor Level Students 


29 

35% 

Master Level Students 


36 

43% 

Doctoral Level Students 


18 

22% 
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Graduates Employed Full Time: 


66 


49% 


Government Employees 

32 

49% 

Industry Employees 

31 

47% 

Higher Education Employees 

3 

4% 

NASA Employees or 

NASA Contractors 

26 

78% 


NASA employment opportunities: 

Graduates Interested 102 76% 


Income Versus Degree (Median Range): 

NASA/NASA Contractors 

Bachelor's Degree $25,00 1-$30, 000 

Master's Degree $30,00 1-$35, 000 

Industry, Government, Higher Education Employees 
(NOT NASA/NASA Contractors) 

Bachelor’s Degree $30,001-$35,000 

Master's Degree $35,00 1-$40, 000 

This analysis reflects the growth in the quality of the Langley Aerospace Summer 
Scholars Program. The program continues to expand and these students are 
providing an excellent pool of qualified candidates for NASA recruitment. Seventy- 
six percent of the respondents indicated they were interested in learning more 
about career opportunities at NASA. 


References: Dr. Samuel E. Massenberg, University Affairs Officer 
Sherry Sullivan, NASA Langley Personnel Division 
Past LARSS Graduate Rosters 


82 



19. How did the LARSS experience influence you? 



